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1. A system, comprising: 

at least one video camera to capture warped panoramic 
video images of a scene and to produce a video stream; and 

a digital processor to receive and process said video 
stream, said digital processor comprising: 

a first processing module to unwrap said warped panoramic 
video images to produce rectilinear video images of the scene, 
wherein said first processing module provides pan, tilt, and 
zoom adjustments to allow for customized viewing of the scene, 
and 

a second processing module to detect and track a person' s 
head in the rectilinear video images and to extract video 
images in the person's view from the rectilinear video images, 

2. The system as in claim 1, wherein said second 
processing module performs an edge detection in the 
rectilinear video images to extract features and an ellipse 
detection to extract possible head images. 

3. The system as in claim 1, wherein said second 
processing module performs Kalman filtering to predict an 
orientation of and track motion of the person's head. 

4. The system as in claim 1, further comprising a 
mechanism to produce a warning signal according to the 
orientation of the person's head. 

5. The system as in claim 1, wherein said digital 
processing comprises a third processing module to process 
images of a face from streaming video images to for face 
recognition. 



BEST AVAILABLE COPY 



WO 2004/004320 



PCT/US2003/020922 



6. The system as in claim 5, wherein said third 
processing module performs a single-frame subspace feature 
analysis on the steaming video images to produce a sequence of 

5 classification results and a sequence of feature vectors and 
then processes said sequences separately to produce face 
recognition outputs . 

7. The system as in claim 6, wherein said third 

10 processing module is configured to apply a majority decision 
rule in processing the sequence of classification results. 

8. The system as in claim 6, wherein said third 
processing module is configured to apply a discrete hidden 

15 Markov model decision rule in processing the sequence of 
classification results. 



processing module is configured to apply a continuous density 
20 hidden Markov model decision rule in processing the sequence'- 
of feature vectors. 

10. The system as in claim 1, further comprising a 
mechanism to measure a facial temperature pattern of a person. 



11. The system as in claim 1, further comprising a 
mechanism for performing a facial affect analysis on a person. 

12. The system as in claim 1, further comprising a 
30 mechanism for performing a speech affect analysis. 



9. The system as in claim 6, wherein said third 
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13. The system as in claim 1, further comprising a 
plurality of video cameras to capture warped panoramic video 
images of the scene at different locations, wherein said 
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interface with pan, tilt, and zoom adjustments to allow for 



customized viewing at each video receiver. 

19. The system as in claim 18, wherein said video servo 
includes a digital process to tracks a change in the scene and 
adjusts filtering in a filtered video stream according to the 
change in the scene. 

20. The system as in claim 18, wherein one video receiver 
includes a PDA. 

21. The system as in claim 18, wherein one video receiver 
includes a laptop computer. 

22. The system as in claim 18, wherein one video receiver 
includes a desktop computer. 

23. The system as in claim 18, where said video camera is 
an omni-directional video camera to capture a 360-degree view 
of the scene. 

24. The system as in claim 18, wherein said digital 
processor includes a video-based face recognition module which 
processes multiple images of a face from a video to perform 
face recognition. 

25. The system as in claim 18, wherein said digital 
processor includes a module that detects and tracks a person's 
head. 

26. The system as in claim 25, wherein said module 
further detects a face orientation of the person. 
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digital processor processes signals from said plurality of 

video cameras and said one video camera to detect and tract 

movement of an object in the scene. 

14. The system as in claim 13 , wherein said digital 
processor performs shadow detection from each video signal 
from each video camera to segment the object from the scene. 

15. The system as in claim 14, wherein said digital 
processor performs a triangulation according to positions of 
the video cameras to produce horizontal positions x, y of the 
object and a height estimation algorithm to produce averaged 
vertical position z of the object. 

16. The system as in claim 15, wherein said digital 
processor registers a track of movement for the object 
according to x and y positions. 

17. The system as in claim 15, wherein the triangulation 
is performed by using an extended N-ocular algorithm. 

18. A system, comprising: 

at least one video camera to capture warped panoramic 
video images of a scene and to produce a video stream; 

a video transmission mechanism to deliver said video 
stream to a plurality of video receivers, said video 
transmission mechanism comprising a video server which filters 
information in said video stream according to security levels 
assigned to video receivers to produce different filtered 
video streams to different video receivers; and 

a digital processor in each video receiver to 
independently process said video stream to unwrap said warped 
panoramic video images to produce rectilinear video images of 
the scene, said digital processor having a user graphic 
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27. The system as in claim 26, wherein said module 

further extracts a video image in the person' s view from the 
video according to estimated face orientation. 

28. The system as in claim 18, wherein said digital 
processor includes a tracking module to detect and track a 
location of an object or a person in real time. 

29. The system as in claim 18, wherein said video 
transmission mechanism includes a wired communication link. 

30. The system as in claim 18, wherein said video 
transmission mechanism includes a wireless communication link. 

31. The system as in claim 18, wherein said video 
transmission mechanism includes a video server that removes 
selected image information from a video signal to send a 
modified video signal to a video receiver. 

32. A system, comprising: 

at least one video camera to capture warped panoramic 
video images of a scene and to produce a video stream; and 

a digital processor to receive and process said video 
stream, said digital processor comprising: 

a first processing module to unwrap said warped panoramic 
video images to produce rectilinear video images of the scene, 
wherein said first processing module provides pan, tilt, and 
zoom adjustments to allow for customized viewing of the scene, 
and 

a second processing module to extract a face from the 
streaming rectilinear video images and to perform face 
recognition on the extracted face. 

33. A system, comprising: 
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at least one video camera to capture warped panoramic 

video images of a scene and to produce a video stream; 

a video transmission mechanism to deliver said video 
stream to a video receiver; and 
5 a digital processor in said video receiver to process 

said video stream to unwrap said warped panoramic video images 
to produce rectilinear video images of the scene , 

wherein said digital processor includes a user graphic 
interface with pan, tilt, and zoom adjustments to allow for 
10 customized viewing of the scene, and said digital processor is 
operable to superimpose a customized video of the scene over a 
digital image. 

34. The system as in claim 33, wherein said digital image 
15 is a map of an area including the scene. 
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